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INTRODUCTION 


Elementary teachers face formidable obstacles when planning and implementing science 
instruction, including inadequate preparation opportunities, lack of resources, and accountability 
pressures. Data from the 2012 National Survey of Science and Mathematics Education bear this 
out (Banilower et al., 2013). Only one-third of elementary teachers have had a course in Earth, 
life, and physical science. And although the vast majority of elementary teachers have had a 
course in science education, only 40 percent report feeling very well prepared to teach science, 
compared to 81 and 77 percent for reading/language arts and mathematics, respectively. When 
asked about specific subjects within science, the percentage of teachers rating themselves as very 
well prepared is even lower; for example, only 17 percent consider themselves very well 
prepared to teach physical science. Instructional resources pose another problem. Although 
several inquiry-oriented curricula are commercially available (e.g., FOSS, Insights, STC), only 
10 percent of elementary classrooms use them. As a result, elementary teachers who are 
committed to inquiry-oriented instruction are too often left to “make it up” on their own. 
Elementary teachers also face overwhelming accountability pressures in mathematics and 
reading; for example, 80 percent of grades 3—5 classes are required to give at least three external 
mathematics assessments (e.g., district benchmark or state tests) each year, and 36 percent 
administer five or more of these assessments. All of these factors have created a situation in 
which science instruction is infrequent and often not of high quality. On average, elementary 
classes spend only 20 minutes per day on science, while devoting 55 minutes to mathematics and 
88 minutes to reading/language arts. A national observation study found that only about one- 
third of elementary science lessons were likely to have a positive effect on students’ 
understanding of science concepts, their ability to carry out their own inquiries, or their 
understanding of science as a dynamic body of knowledge (Weiss, Pasley, Smith, Banilower, & 
Heck, 2003). Data from the 2012 National Survey indicate that elementary science instruction 
continues to be based predominantly on lecture/discussion, with limited use of hands-on 
activities and evidence to support scientific ideas (Banilower et al., 2013). 


The expectations for elementary science instruction were recently raised to a new level. The 
Next Generation Science Standards ([NGSS] NGSS Lead States, 2013) are the latest in a series 
of college and career-ready standards released over the last few years. Together with the 
Common Core State Standards in Reading and Mathematics (National Governors Association 
Center for Best Practices & Council of Chief State School Officers, 2010a, 2010b), they put forth 
an ambitious vision of what students should know and be able to do in these fields as a result of 
K-12 education. Twenty-six states partnered in developing the NGSS, and 18 states have 
formally adopted them to date, suggesting the standards will influence the system over the next 
several years. 


To realize the vision of excellent science education for all students portrayed in the NGSS, 
elementary teachers will need to draw on a wide variety of knowledge. Prominent educators and 
researchers have proposed the existence of a professional knowledge base for teaching, similar to 
the specialized knowledge bases for medicine and law (Grossman, 1990; Hiebert, Gallimore, & 
Stigler, 2002; Hill, Rowan, & Ball, 2005; Shulman, 1986a, 1986b). Efforts to articulate the 
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components of such a knowledge base have been underway for over two decades. Some 
constituent knowledge forms, such as disciplinary content knowledge, are fairly well understood 
and widely accepted as necessary, but not sufficient, for effective teaching (e.g., Heller, Daehler, 
Wong, Shinohara, & Miratrix, 2012). 


Perhaps the most widely recognized form of specialized knowledge for teaching—and arguably 
the one with the most potential for helping teachers overcome knowledge-related obstacles—is 
“pedagogical content knowledge” (PCK), which Shulman (1986a, 1986b) described as an 
amalgam of pedagogical knowledge (general teaching knowledge) and content knowledge 
(knowledge of a specific discipline). An oft-cited example is knowledge of an effective strategy 
for teaching a particular concept; for example, having students slide an object on progressively 
smoother surfaces to construct an understanding of the idea that an object in motion tends to 
remain in motion in a straight line unless a force acts on it. After years of research, there is an 
emerging consensus that PCK is indeed important for science teachers. Magnusson, Krajcik, and 
Borko (1999) developed a model of PCK that has strongly influenced conceptualizations of what 
constitutes PCK in science as well as other disciplines. The model describes discrete forms of 
PCK, including knowledge of instructional strategies, knowledge of students’ understanding of 
science, and knowledge of science curriculum. Since the introduction of the Magnusson et al. 
model, PCK-related research in science education has accelerated. Still, interpretation of the 
construct varies widely. Some researchers contend that PCK is highly personal or idiosyncratic 
(Loughran, Mulhall, & Berry, 2008; Park & Oliver, 2008; Van Driel, Verloop, & de Vos, 1998), 
developed over time through teaching experience (Appleton, 2008; Friedrichsen et al., 2009; 
Nilsson, 2008). Others assert that PCK can accumulate and form a collective body of knowledge 
independent of the teacher (Fernandez-Balboa & Stiehl, 1995; Veal & MaKinster, 1999). 


At a weeklong invitational conference on science PCK in 2012, attended by the lead author, a 
group of researchers agreed on a revised model of PCK that acknowledges both collective and 
personal aspects of PCK (Gess-Newsome, 2015). In this model, shared, or collective, PCK is 
referred to as topic-specific professional knowledge (TSPK); however, for our purposes, we refer 
to it as collective PCK, or C-PCK. Hypothesized relationships among these and other forms of 
knowledge are shown in Figure 1. 


As illustrated in the model, discrete professional knowledge bases are the foundation for C-PCK. 
Examples of C-PCK include an instructional strategy that has been found through empirical 
studies to be effective for teaching a specific idea, or recognition of a conceptual difficulty found 
through assessment studies to be prominent among elementary students. This knowledge can be 
applied by teachers to their own unique settings and for their own purposes. As teachers take up 
C-PCK—through reading, professional development experiences, discussions with colleagues, 
reflecting on their practice—and use it in their teaching, it becomes personal PCK. 
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PCK Model 


Teacher Professional Knowledge Bases 


Assessment Pedagogical Content Knowledge of Curricular 
Knowledge Knowledge Knowledge Students Knowledge 


| 


Topic-Specific Professional Knowledge (TSPK) 
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student understandings, science practices, and habits of ming 


Amplifiers and Filters: teacher beliefs, 
orientations, prior knowledge, and context 


Classroom Practice 


qn, Classroom Context 
eS d (Curriculum, etc. 


Figure 1 


C-PCK has the potential to help elementary teachers overcome knowledge-related obstacles to 
science teaching in several ways. Most importantly, C-PCK provides a rich resource for helping 
teachers incorporate what is known about effective teaching of a topic into their instruction (see 
Figure 2). C-PCK can be a valuable instructional planning resource or it can, for example, be the 
focus of discussion in a teacher study group or professional learning community. Another high- 
leverage use of C-PCK is in instructional materials development (Banilower, Nelson, Trygstad, 
Smith, & Smith, 2013). For example, instructional materials can be infused with C-PCK, 
making the knowledge directly available to teachers, both through the design of activities for 
students and in the supports for teachers that accompany those materials, making the 
instructional materials “educative” for the teacher (Davis & Krajcik, 2005). Similarly, teacher 
educators and professional development providers can use C-PCK to craft and provide topic- 
specific support for pre-service and in-service teachers. 
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C-PCK Theory of Action 
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Figure 2 


There is a common perception that C-PCK is widely available. Although C-PCK exists and has 
been compiled in a few topics (e.g., empirical research abounds for force and motion), many, 
perhaps most, science topics are not well researched. Even a brief search of the literature 
illustrates the lack of easily accessible C-PCK in many topics. In addition, the literature that 
does exist is not organized for use. Therefore, teachers are left to winnow through an 
unorganized literature of sometimes suspect quality, whether it is empirical research or wisdom 
of practice. To illustrate, a search of peer-reviewed literature in the ERIC database using the 
terms “ecosystems” and “elementary education” returns 77 results, which seems encouraging. 
However, only a few of the articles are relevant for an elementary teacher planning a unit or 
lesson on ecosystems. The following articles appear at the top of the results list (bold text added 
for emphasis): 


e The Lifelong Learning Ecosystem in Korea: Evolution of Learning Capitalism? 

e Writing to Learn Ecology: A Study of Three Populations of College Students 

e Ecological Psychology: Potential Contributions to Social Justice and Advocacy in 
School Settings 

e Seeding Evolutionary Thinking by Engaging Children in Modeling Its Foundations 


At the opposite end of the spectrum, a search for a more focused concept—“‘interdependence”’ 
within ecosystems at the elementary level—returns no results. Similar searches in Google 
(which teachers are probably more likely to use than ERIC) result in several hundred thousand 
results, but there is no way to vet the quality of these resources. 


The vast majority of elementary teachers (92 percent) are responsible for teaching multiple 
subjects (Banilower et al., 2013). Their preparation time is severely limited, and they feel 
inadequately prepared to teach science. Elementary teachers do not have the time, and many do 
not have the necessary background, for extensive literature searches. The immediate goal of our 
study—titled Knowledge Assets to Support the Science Instruction of Elementary Teachers 
(ASSET)—1s to test a method for collecting and synthesizing PCK, then assemble that 
knowledge in a C-PCK resource that will transform teacher practice. The larger goal is to create 
an efficient, scalable method that can be applied to a broad array of science topics. To these 
ends, our project is exploring the following questions: 


1. What are the strengths and weaknesses of a C-PCK collection-and-synthesis method? 
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2. What factors must be taken into account in applying the C-PCK collection-and- 
synthesis method across topics? 

3. What affordances and limitations does the C-PCK resource present for teachers 
primarily, and also for teacher educators and instructional materials developers? 

4. How does access to C-PCK affect teachers’ planning and instruction? 


Reporting on our work to date, this paper focuses exclusively on the first two research questions. 


METHODOLOGY 


To test the robustness of the C-PCK collection-and-synthesis method across topics, our study 
(titled Knowledge Assets to Support the Science Instruction of Elementary Teachers, or ASSET) 
focuses on two disciplinary core ideas in the NGSS at 5" grade: (1) Structure and Properties of 
Matter (which we refer to as the Small Particle Model, or SPM), and (2) Interdependent 
Relationships in Ecosystems (which we refer to as Interdependence). The work of identifying 
and synthesizing C-PCK proceeded on two fronts. First, we reviewed empirical research studies 
and synthesized information on student understanding and topic-specific instructional strategies. 
Second, we collected and synthesized practice-based knowledge about instructional strategies 
reported to be effective in developing student understanding of the designated topics. We 
anticipated that practice-based knowledge, a long-overlooked resource, would be the primary 
source of C-PCK for some topics in the NGSS. We collected practice-based knowledge through 
literature searches and through surveys and interviews of teachers, science education researchers, 
and instructional materials developers, each with topic-specific expertise. 


We recognized that before work could begin on these fronts, we needed to be very clear about 
the science ideas included in the two topics. Therefore, the first step in the process was to 
unpack and clarify the two topics (SPM and Interdependence). These content clarifications were 
reviewed for content accuracy by a university scientist or science educator with expertise in the 
topic. The clarifications were then revised based on their feedback. The final versions of each 
clarification are shown in Figures 3 and 4. 
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Small Particle Model 


NGSS Disciplinary Core Idea (5-PS1.A partial) 

1. Matter of any type can be subdivided into particles that are too small to see, but even then the matter still 
exists and can be detected by other means. A model showing that gases are made from matter particles that 
are too small to see and are moving freely around in space can explain many observations, including the 
inflation and shape of a balloon and the effects of air on larger particles or objects. (PE 5-PS1-1; DCI 
PS1.A) 


2. The amount (weight) of matter is conserved when it changes form, even in transitions in which it seems to 
vanish. (PE 5-PS1-2; DCI PS1.A) 


ASSET Clarification 
1. All matter is composed of particles that are too small to see even with a microscope. 
a. The particles have empty space between them. 
b. The particles are in constant random motion. 


2.A particle model of matter can be used to describe and explain important phenomena, including what 
happens when a liquid evaporates and when a solid dissolves in a liquid. The model can also explain why 
matter is conserved when it changes form. 


Figure 3 
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Interdependence 


NGSS Disciplinary Core Idea (5-LS2.A) 
The food of almost any kind of animal can be traced back to plants. Organisms are related in food webs in 
which some animals eat plants for food and other animals eat the animals that eat plants. Some organisms, 
such as fungi and bacteria, break down dead organisms (both plants or plants parts and animals) and therefore 
operate as “decomposers.” Decomposition eventually restores (recycles) some materials back to the soil. 
Organisms can survive only in environments in which their particular needs are met. A healthy ecosystem is 
one in which multiple species of different types are each able to meet their needs in a relatively stable web of 
life. Newly introduced species can damage the balance of an ecosystem. 


ASSET Clarification 
1. The food of almost any kind of organism can be traced back to producers such as plants and algae. 
a. Food provides organisms the materials and energy they need to grow and function. 
b. Producers make their own food inside themselves using (1) energy from the sun and (2) matter from air 
and water. 


2. Organisms in ecosystems are related in food webs 
a. Consumers get their food by eating other organisms. Some consumers eat producers. Some consumers 
eat other consumers. 
b. Decomposers, such as bacteria, fungi and earthworms, are consumers that break down dead organisms 
(or parts of organisms). 
c. Decomposition eventually restores (recycles) some materials back to the environment, making necessary 
materials available to producers. 


3. Organisms can survive only in environments in which their particular needs are met. Environmental 
conditions include, but are not limited to, light, temperature, moisture, amount of oxygen, nutrient 
availability, and salinity. 


4.In a healthy ecosystem, the needs of multiple types of organisms are met in a relatively stable web of life. 
5. Natural events and human activity can change the balance or stability of an ecosystem. When the balance, 


or stability, of an ecosystem changes, the opportunities for different types of organisms to meet their needs 
can increase or decrease. 


Figure 4 


Approach #1: Collection of Empirical Knowledge 


Project researchers conducted a review of empirical studies (and syntheses of empirical studies) 
for each topic, which consisted of searching the literature (e.g., refereed journals, conference 
proceedings, or published books), reviewing studies, and synthesizing findings. Researchers 
began the literature search by identifying a list of key search terms, such as “particle model of 
matter,” “structure of matter,” and “teaching methods” for SPM. The full list of key terms for 
each topic can be found in Table 1. Individually, these terms returned a broad spectrum of 
results from the search engines (ERIC and Google Scholar among them), so many of the terms 
were used in combination in order to narrow the literature to that which relates to elementary 
teaching. Two examples include, “particle model AND elementary students” and “changes in 
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state AND misconceptions'.” To be included in the review, a study had to meet the following 


criteria: 


e Reported in a peer-reviewed journal, peer-reviewed conference proceedings, or an 


edited book; 


e Included K—8 students in the study sample; 

e Was a systematic empirical study (strictly theoretical pieces were not included); and 

¢ Could not be a literature review only (however, the bibliographies of these articles 
were used to identify primary sources). 


Concept formation 
Conservation of mass 


Instructional design 
Instructional strategies 


Particulate nature of matter 
Particulate theory 


Table 1 
Key Search Terms 
SPM 
Changes in state Elementary education Misconceptions Science education 
Chemistry Elementary school science Models Scientific concepts 
Cognitive development Evaporation Particle model of matter States of matter 
Computer simulations Gaseous state Particulate Structure of matter 


Student ideas 
Student thinking 


Condensation Learning progressions Pedagogy Teaching methods 
Interdependence 

Activities Ecosystem Elementary education Lessons 

Concepts Ecosystem imbalance Elementary student Predator 

Curriculum Ecosystem interdependence Food web Prey 

Decomposer Ecosystem interdependencies | Instruction Producer 

Decomposition Ecosystem misconceptions Learning Student knowledge 

Ecology Ecosystems Lesson Student thinking 

Ecology misconceptions Elementary Lesson plans Understanding 


Articles were initially screened by reading only the abstract. Those that appeared to meet the 
review criteria were saved in a reference management program. The project team also created a 
list of tags to be applied as the articles were read more carefully. These tags were used to filter 
the collection. For example, articles that were found to focus on high school students, pre- 
service teachers, or in-service teachers (but not K—8 students) were excluded from the final 
collection of literature. 


Once the literature pool was finalized, researchers began coding the kinds of PCK in each study 
for teaching the topic to upper elementary students. The coding scheme was both a priori, based 
on the Magnusson et al. (1999) model of PCK, and emergent. Magnusson et al. describe discrete 
forms of PCK, including knowledge of instructional strategies, knowledge of students’ 
understanding of science, and knowledge of science curriculum. In some cases, we elaborated 
on these forms, for example adding misconceptions and learning progressions as categories of 
knowledge of student understanding. 


' In this paper, “misconception” is used to denote any idea that conflicts with accepted scientific ideas about a 
phenomenon, acknowledging that such ideas may represent a productive step in a student’s learning progression. 
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Because the articles in the pool varied in the quality of the research designs, the research team 
added a confidence rating for each piece of PCK coded. This rating was intended to reflect the 
reliability and generalizability of the PCK. For example, if an article claimed that upper 
elementary students are likely to have a particular misconception, the rating conveyed how 
confident we were that the claim is true. To determine the confidence rating, we adapted the 
Standards of Evidence (SoE) review (Heck & Minner, 2009). The SoE review assesses the 
extent to which key components of the research are documented and judges support for findings 
considering each question that the publication addresses. Key in the assessment of rigor is the 
consideration of multiple aspects of internal validity. We selected a subset of indicators in order 
to create an abbreviated review process, focusing on five factors: 


1. Sample size; 

2. Appropriateness of analyses; 

3. Validity and reliability of research instruments; 
4. Appropriateness of generalizations; and 

5. Potential for investigator bias. 


Additionally, we looked at the alignment of the purpose of the research and the coded PCK when 
determining the confidence rating. If a study was not designed to identify student 
misconceptions, but student misconceptions were identified incidentally, the coded 
misconceptions received a low confidence rating. However, if the same item of PCK was 
extracted from many articles in the literature pool, even if it received a low confidence rating 
each time, the item would receive a high rating overall based on the accumulation of evidence. 


A codebook was developed to provide descriptions of each code, rules for when a code should be 
applied, and explanations of how to determine confidence ratings. The codebook is included in 
the Appendix. The project team iteratively reviewed articles, applied the coding scheme 
(comparing codes and confidence ratings), and refined the codebook. 


Approach #2: Collection of Practice-based Knowledge 


Given the anticipated limits of empirical literature, and the great need for C-PCK, we decided to 
tap practice-based knowledge as well. First, we searched for and reviewed practitioner literature; 
for example, articles that appear in Science and Children. In contrast to systematic studies 
reported in empirical literature, we reviewed practitioner articles for reports of practitioners’ 
experiences using particular instructional strategies to teach a topic. To be included in the 
review, an article had to meet the following criteria: 


e Provided evidence of peer review; 

e Included K-—8 students in the instructional focus; 

e Focused on instructional experiences or findings aligned with at least one of the 
ASSET clarification ideas; 

e Had minimal or no cost; 

e Was not dated (e.g., did not include inaccurate statements or culturally insensitive 
language); and 
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e Had a strong likelihood of being accessible by most educators. 


We applied additional criteria to articles for Interdependence to ensure broad applicability of 
PCK to a variety of instructional contexts. Specifically, we did not review articles focusing on 
specific types of ecosystems (e.g., longleaf pine, coral reef), or particular types of organisms 
(e.g., insects, duckweed, polar bear) when connections to general ecosystem concepts were not 
evident. Likewise, we excluded many articles that focused on photosynthesis without 
emphasizing its role in ecosystems. 


The second strategy for collecting practice-based knowledge played out much differently than 
expected. We planned to construct expert panels and elicit wisdom of practice via an online 
panel format, consistent with Loughran et al.’s assertion that efforts to collect and synthesize 
practitioners’ PCK must work at both individual and collective levels (Loughran, Mulhall, & 
Berry, 2004). The panels were envisioned as a variation of the Delphi panel strategy, which has 
been used in many fields to elicit knowledge from expert practitioners (Edmunds, Garratt, 
Haines, & Blair, 2005; Hauck, Kelly, & Fenwick, 2007; Kingsley & Waschak, 2005; Muijs, 
2006; Roff, McAleer, & Skinner, 2005; Wen & Shih, 2008). Panelists would respond in writing 
to a set of questions (described below) about the topic. Responses would then be analyzed and 
synthesized to generate draft statements of C-PCK, and iterative panel rounds would be 
conducted to clarify areas of disagreement. 


The panels (one for each topic) were to consist of 12 experts: 6 master elementary teachers, 3 
teacher educators, and 3 instructional materials developers. Each panelist was to have expertise 
in the relevant topic. The master teachers would be individuals who had extensive experience 
teaching the target topic, had experience supporting other teachers in the topic, and were 
recognized by their peers for their excellence. As planned, we selected them primarily from 
recipients of and finalists for the Presidential Award for Excellence in Mathematics and Science 
Teaching (PAEMST). Teacher educator panelists were to include both college/university faculty 
and professional development providers, and instructional materials developers would include 
individuals who had been involved in developing materials in the two topics at the elementary 
level. For reasons described in the Findings section, we surveyed and interviewed all types of 
practitioners, but we did not implement iterative rounds. In addition, we collected data from 
more teachers than planned and from fewer teacher educators than anticipated. 


The interview and survey questions for panelists were adapted from the Content Representation 
(CoRe) interview, developed by researchers at Monash University in Australia (Loughran et al., 
2004) and used widely in studies of teacher knowledge. Adapted CoRe interview questions for 
practitioners are listed in Figure 5. The survey followed a similar structure. 
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Sample Interview Questions for Practitioners 


1. What do you think 4-6" graders need to already know or understand in order to learn [this content]? 


2. What ideas, naive conceptions, or misconceptions do you think 4-6" graders have that would make it 
difficult for them to learn about [this content]? 


3. What kinds of prompts or strategies might you use to get students to express their initial ideas about [this 
content]? 


4. Can you think of anything about students’ development at this age (10-11 yrs old) that may make [this 
content] difficult for them to learn? 


5. Can you think of experiences that kids this age may have had—whether in school or out of school—that 
you could draw on to help them understand [this content]? 


6. How would you go about teaching [this content] to 4""-6" graders? What approaches, activities, or 
strategies would you use? 


7. Can you think of other resources (e.g., readings, simulations, etc.) for helping 4""-6" graders develop an 
understanding of [this content]? 


8. What else, if anything, should someone teaching [this content] know or consider that you have not already 
talked about? 


Figure 5 


Synthesizing Findings from the Two Approaches 


We are currently synthesizing findings from empirical literature and practice-based knowledge 
(from practitioner literature and practitioners’ responses) about the teaching and learning of 
SPM. (We are still collecting practice-based knowledge for Interdependence and have not yet 
started to synthesize across empirical and practitioner sources.) As described earlier, the goal is 
to use practice-based knowledge to fill in gaps in the empirical literature. Ultimately, the 
synthesized C-PCK will be made available to the field through a project website. The 
comprehensive process is summarized in Figure 6. 
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C-PCK Collection and Synthesis Method 


( Topic Specification 


Clarification of Idea 


Collection of Empirical Knowledge Collection of Practice-based Knowledge 
ASSET collects and reviews | ASSET collects and reviews Panelists respond to 
empirical literature. practitioner literature. CoRe instrument. 


ASSET synthesizes findings ASSET synthesizes findings 
from empirical literature. from practitioner literature. 


ASSET synthesizes 
responses. 


| ASSET drafts C-PCK website. | 


A 
( ASSET Advisory Board reviews draft website. ) 


A 


( ASSET revises C-PCK website for teacher use. ) 


Figure 6 


FINDINGS 


As mentioned above, we selected two 5" grade DCIs from the NGSS, one from physical science 
(SPM) and one from life science (Interdependence). This design feature was intentional, 
allowing us to explore the possibility that the types and amount of PCK available might vary by 
topic. In retrospect, selecting SPM as one of the topics was probably a mistake. The NGSS 
advocate a particle approach to this topic, but without invoking atoms and molecules. 
Historically, the content has not been addressed in this way. The National Science Education 
Standards (National Research Council, 1996) did not introduce the particle model of matter 
(using atoms and molecules) until grades 9-12, justifying the placement as follows: 


It can be tempting to introduce atoms and molecules or improve [grades 5-8] 
students’ understanding of them so that particles can be used as an explanation for 
the properties of elements and compounds. However, use of such terminology is 
premature for these students and can distract from the understanding that can be 
gained from focusing on the observation and description of macroscopic features 
of substances and of physical and chemical reactions. (National Research 
Council, 1996, p. 149) 


The Benchmarks for Science Literacy (American Association for the Advancement of 
Science, 1993) introduced the particle model in middle grades (6-8), but no earlier. The 


Horizon Research, Inc. 12 April 2016 


Benchmarks state that, “By the end of gm grade, students should have sufficient grasp of 
the general idea that a wide variety of phenomena can be explained by alternative 
arrangements of vast numbers of invisibly tiny, moving parts” (American Association for 
the Advancement of Science, 1993, p. 77). 


As noted above, SPM has traditionally been taught using atoms and molecules, as well as 
atomic/molecular mechanisms, something the NGSS explicitly discourage: “Assessment does 
not include the atomic-scale mechanism of evaporation and condensation or defining unseen 
particles” (NGSS Lead States, 2013, p. 43). In the discussion of findings below, it is important 
to bear in mind that although we are applying our methodology to two topics to test its 
generalizability, one of those topics may in fact have been an outlier in the sense that it is new to 
elementary grades instruction. 


Searching the empirical literature yields PCK that is predominantly about student thinking 
related to the content. 

The search-and-screening process yielded 44 empirical articles and book chapters for SPM and 
55 for Interdependence (bibliographies are included in the Appendix). Although these numbers 
may seem low, it is important to remember that studies had to meet specific criteria to be 
included, as described earlier. From these, we coded approximately 300 instances of PCK for 
each topic. The vast majority of these instances were in the category of student thinking (e.g., 
misconceptions, alternative conceptions, cognitive precursors, and learning progressions). 
Examples of coded PCK include: 


e For the Small Particle Model 

o Elementary-age students often think of matter as continuous rather than 
particulate.(Nakhleh & Samarapungavan, 1999; Renstrém, Andersson, & Marton, 
1990) 

o Students who acknowledge a particulate nature of matter sometimes think that a 
substance (e.g., air or water) fills the empty spaces between the particles. (Lee, 
Eichinger, Anderson, Berkheimer, & Blakeslee, 1993; Novick & Nussbaum, 
1978) 

o Students who acknowledge particle movement often think it occurs only when 
external forces are applied. (Lee et al., 1993) 


e For Interdependence 

o Students often think that organisms can be affected only by those that are directly 
linked to them in food webs. (Grotzer, 2009; Ozkan, Tekkaya, & Geban, 2004) 

o Many students think that dead organisms (or organism parts) are absorbed by soil 
or simply disappear, and thus see no role for decomposers in ecosystems. (Hogan 
& Fisherkeller, 1996) 

o Students who acknowledge that plants make their own food often do not realize 
that they need light, water, and air to do so. (Helldén, 1998) 


From these studies, we were also able to code a large number of prompts that researchers used to 
elicit student thinking (1.e., they were not designed for instructional purposes but might be used 
as such). For example: 
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e For the Small Particle Model 
o Imagine you are Superman and can see inside a material (water, clay, stone, 
wood, etc.) with your x-ray vision. Draw what you see with as much detail as 
possible. (Acher, Arca, & Sanmarti, 2007, p. 406) 
o When you fan yourself with your hands, although your hands do not touch your 
face, you feel that something hits your face. In your opinion, what might this be? 
(Durmus & Bayraktar, 2010, p. 501) 


e For Interdependence 
o Plants need food for growth, just as animals do. Where does their food come 
from? (Smith & Anderson, 1984, p. 693) 
o One student I talked to said that bigger animals eat small animals in a food web. 
Do you agree or disagree, and why? (Grotzer, 2009, p. 26) 
o Where does soil come from? (Helldén, 1998, p. 6) 


Nearly all the studies that met our criteria can be divided into two categories with respect to their 
main focus—examinations of student thinking and instructional efficacy. For SPM, more than 
three quarters of the documents focused mainly or exclusively on student thinking. For 
Interdependence, the main focus was more evenly divided with only slightly fewer targeting 
instructional efficacy than student thinking. For example, we found empirical studies about the 
efficacy of fieldwork strategies for teaching about Interdependence. We also found studies 
comparing alternative means for teaching about food webs, including some studies that explored 
students’ thinking in response to teaching approaches that emphasized either concrete 
interactions among organisms or more abstract system-level thinking. Finally, a few studies 
examined the utility or efficacy of role-play or other simulation-based strategies. 


We anticipated a focus on student thinking in the empirical literature, and it motivated our plans 
to incorporate practice-based knowledge. However, we were surprised by the heavy emphasis 
on student thinking in the research on both topics. 


The PCK available in practitioner literature is heavily dependent on the topic. 

As mentioned above, SPM has typically been introduced after elementary grades and has 
typically incorporated atoms and molecules. Our search uncovered only a handful of articles 
related to teaching SPM to elementary students, and we believe it is because the topic has not 
broadly been taught as advocated in the NGSS. We did find practitioner literature on properties 
and states of matter, but the emphasis was on description rather than explanation. We did not 
find instances in which SPM was invoked to explain properties, changes or state, or other related 
phenomenon (e.g., conservation of matter during phase change or dissolving). 


In contrast, we found approximately 70 pieces of practitioner literature for Interdependence. The 
vast majority of these focused on ideas for teaching the content. About half described a coherent 
unit of instruction for content that included, or substantially overlapped with, the ASSET 
clarification ideas. Activities within many of these units depended on visits to particular types of 
ecosystems or even specific locations, and may therefore have less general utility. About one- 
third described a single activity that was intended to be implemented as a discrete lesson (e.g., 
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one or two class periods). We often found these descriptions in practitioner-oriented periodicals 
such as Science Scope or Science and Children. Eight of the practitioner pieces described an 
“approach” (e.g., incorporating simulations into instruction) and used an activity or activities as 
an example to illustrate that approach. Just 1 of the 70 pieces had an exclusive focus on student 
thinking and did not center on instruction. Our search surfaced many documents from the 1970s, 
perhaps reflecting a time of great emphasis on environmental education more generally. 


Identifying expert teachers was much more difficult than anticipated. 

The first challenge in identifying expert teachers was defining the criteria for expertise. The goal 
was to recruit teachers with expertise teaching the topic (SPM or Interdependence), but evidence 
of such expertise was elusive. Instead, for SPM, we recruited teachers who had documented 
evidence of expertise as science teachers and who had at least some experience teaching the 
topic. For this purpose, we used the network of recipients of, and finalists for, the PAEMST. 
One of the individuals who reviewed SPM ideas for the project is a PAEMST awardee and 
offered to announce the study to various PAEMST social media groups. However, so few 
individuals volunteered for the study that we were unable to select a subset based on expertise 
demonstrated through materials submitted (see the finding below related to the CoRe 
questionnaire). Each of the individuals reported teaching about SPM, but it became clear in 
reviewing their questionnaire responses and submitted materials that the core ideas were not a 
central focus of their instruction about matter, and sometimes not a focus at all. 


As a result, we decided to cast a broader net and collect more information about teachers before 
asking them to serve as experts. Using an email list of approximately 600 grades 4—5 teachers in 
the U.S., we sent an invitation to register for the study. The registration form included several 
questions designed to help identify teachers with substantial experience teaching the topic (a 
proxy for expertise), among them: 


Years of teaching experience; 

Familiarity with the NGSS; 

Amount of instructional time spent on the topic; 

Number of times the individual had taught the topic; and 

Attention to ideas within the topic (e.g., within SPM, registrants indicated whether 
they focused on the idea that particles are in constant random motion). 


Well over 100 teachers registered for each topic, allowing us to select those who appeared, based 
on their answers, to have substantial experience teaching the topic and also to include a focus on 
the ideas within the topic. We subsequently collected data about teachers’ instruction from a 
much larger number of individuals than we planned, with the idea that PCK would be validated 
by the frequency with which it appeared in their data rather than relying on evidence of 
individual teachers’ expertise (e.g., being the recipient of an award). The topic-dependent nature 
of practitioner literature appears to also be evident in teachers’ firsthand accounts of their PCK. 
Data collected from teachers about their instruction related to SPM suggest little actual 
experience teaching the content, despite their indications to the contrary when they registered for 
the study. Rather, in their responses, we see an emphasis on properties of matter without a focus 
on the underlying explanatory power of SPM. For example, many teachers described student 
thinking and their instruction for states of matter and changes of state, but their descriptions did 
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not indicate that addressing particles or particle behavior was an important aspect of instruction. 
In contrast, the data collected so far from teachers about their instruction on Interdependence 
indicate that they actually do teach the ideas as envisioned in the NGSS. For example, teachers 
provided evidence that their instruction focused on such ideas as producers using light, air, and 
water to make their own food; transfer of matter and energy in food webs; the role of 
decomposers in recycling matter within ecosystems; and changes for one population in a food 
web affecting all others. 


Getting teachers to articulate their PCK has been more challenging than anticipated. 

Despite literature suggesting that teachers may struggle to articulate their PCK (e.g., Loughran et 
al., 2004), we were surprised at the magnitude of the difficulty. The CoRe questions (see Figure 
5) are perfectly reasonable questions, but teachers rarely have an opportunity to consider them in 
conversation with others. They require a level of reflection that teachers are clearly capable of 
but have little or no reason to engage in. The questionnaire version of the CoRe questions did 
yield some PCK (more so for Interdependence than for SPM), but we have found ourselves 
wanting to follow up on responses with questions like, ““Why that activity at that time?” and 
“How does that activity relate to the rest of your instruction on this topic?” 


We believe we have had more success with interviews based on the CoRe, which is the format 
for which it was originally designed. That said, the interviews are difficult to conduct well, 
requiring advanced interview skills and a deep familiarity with the content and PCK for the 
topic. We found that an interview question activates a tacit and not clearly organized network of 
knowledge. Consequently, the interviewer has to be able to follow the teacher wherever he or 
she goes until the network is fully articulated, at least to the best of the teacher’s and 
interviewers’ ability. Teachers often strayed into content that is clearly not a part of the ideas as 
laid out in the NGSS; for example, teachers began discussing their instruction about chemical 
bonding. 


In interviews, evidence of teachers’ lack of experience articulating their PCK is seen in their 
frequent use of the phrase, “We talk about...,” referring to their instruction about a particular 
topic or idea. When interviewers probed on this response, it was clear that sometimes teachers 
literally talked about the ideas with their students. For example, one teacher described talking 
with the students about what would happen if a bottle of perfume were opened in the classroom, 
but without ever actually bringing a bottle of perfume to class and opening it. For other teachers, 
“we talk about...” was just a placeholder for a variety of instructional approaches that went 
beyond discussion. 


Finally, the generative function served by asking teachers to articulate their PCK is worth noting. 
As teachers respond, tacit knowledge can become explicit, and teachers sometimes form new 
PCK for themselves in describing their instruction and the underlying assumptions. This fact 
makes representing a teacher’s PCK particularly difficult, as it may not be stable. It was not 
uncommon for teachers to answer, “I’ve never thought about that” to some of the CoRe 
questions, but that did not deter them from elaborating. 
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Instructional materials developers may be particularly rich sources of practitioner PCK. 

To date, we have interviewed three developers of instructional materials for SPM. Although the 
sample is small and almost certainly not representative, we have found them much more able to 
articulate their PCK and, consequently, richer sources of PCK. Each of the individuals has 
substantial experience developing materials and supporting teachers in using those materials. 
They have clearly thought deeply about the kinds of knowledge teachers need, both in the 
process of developing materials and in designing professional development experiences. In 
short, these individuals have had both reason and opportunity to articulate their PCK. An 
example of each individual’s PCK follows: 


e One developer reported using the terms “atoms” and “molecules” with 5 _ grade 
students. When questioned about this decision, the developer explained that students 
were more comfortable with those words than “particles.” The students did not 
understand the scientific significance of the terms, but they used them spontaneously 
when talking about “water molecules” for example. In response, the developer began 
using “atoms” and “molecules” as synonyms for “particles” in order to facilitate 
student understanding of SPM. There was no expectation that students understand 
the scientific meanings of the terms. 


e One developer described students’ thinking about vacuum as a particularly rich 
context for developing the small particle model. Students have a deeply held notion 
that a vacuum is associated with “sucking,” when SPM explains that in fact air 
particles (or those of some other gas) are pushing. Using a computer simulation, 
students can visualize what happens when air particles are added to or removed from 
a system. 


e When asked what students need to know or understand before engaging with SPM, 
one developer stressed the importance of understanding the idea of inference—t.e., 
making claims about things that cannot be seen from things that can. The developer 
explained that inference is critical for learning about SPM because students will never 
be able to see the particles. Instead, they will have to infer from phenomena they can 
see—for example, a balloon shrinking as the air around it cools or the conservation of 
matter when sugar dissolves in water. 


In each of the instances, there is an element of explicitly purposeful thinking that we rarely saw 
in interviews with teachers. That is, the developers could describe what they did in their 
materials as well as why, and the why in each case was based on student thinking. 


An appropriate grain size for sharing C-PCK is not clear. 

Our ultimate goal is to make the PCK we collect available to teachers, teacher educators, and 
instructional materials developers. For some types of PCK, we anticipate sharing the knowledge 
in essentially the same form we found it in the literature or learned about it from practitioners. 
For example, with regard to SPM, we know from multiple sources that 5m grade students are 
likely to struggle conceptualizing the existence and behavior of particles that are far too small to 
see. We also know that a particularly common approach to revealing student thinking about this 
aspect of the model is to ask students to imagine and draw what they would see if they looked at 
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something (e.g., a glass of water) with very strong magnifying glasses. Similarly, we know that 
instruction often incorporates students interacting with on-line simulations—for example, the 
PhET simulations (https://phet.colorado.edu/). We anticipate sharing these instances of PCK 
essentially as is. Less clear is what to do with an intact lesson plan or curriculum unit, 
particularly if copyright is involved, which is often the case. On one hand, these resources may 
be just the ones teachers need. On the other hand, they may be inaccessible (e.g., a lesson plan 
from a journal that requires a subscription for access) or prohibitively expensive (e.g., a module 
from a kit-based commercial curriculum). Our current thinking is that we will not share lesson- 
or unit-level instantiations of PCK but rather the more general (but still topic-specific) principles 
on which they are based. 


CONCLUSIONS AND IMPLICATIONS 


The ASSET project was based on an assumption that the PCK available in empirical literature 
for any given topic would be spotty and that we would be able to fill in the gaps with practitioner 
knowledge. For SPM, the empirical literature contains substantial knowledge about how 
students think about the structure of matter but very little about how to teach the ideas to upper 
elementary students. In Interdependence, the empirical literature was more balanced but still 
biased toward student thinking. We are not certain how much of the difference in emphasis 
between the two topics is due to the newness of SPM in upper elementary grades. 


In addition to our empirical literature review, a survey of curriculum materials (e.g., STC, 
Insights, and FOSS) and pre-NGSS state and national standards also supports the claim that SPM 
is uncharted territory for many if not most elementary teachers. As such, the topic presents 
unique challenges in terms of available PCK. The newness of the topic also presents a 
formidable challenge for teachers in terms of availability of instructional materials. Of course, 
because of the emphasis on 3-dimensional learning, the NGSS present major challenges for all 
teachers, but SPM is unfamiliar content in addition. We wonder how many other topics in the 
NGSS are like SPM in this regard and how teachers can best be supported in teaching them. 


As we became aware of the widespread unfamiliarity with SPM, we began considering 
experience teaching the targeted content as a factor when selecting study participants. Despite 
such considerations and subsequent changes to our recruitment process, our efforts to 
complement PCK from empirical literature have so far had limited success. In particular, we, 
like others, have found it difficult to draw PCK out of teachers. The teachers we interviewed 
seemed to fall in three categories. Teachers in the smallest of these categories were able to 
describe how they teach a topic and why they employ particular strategies. Another group could 
explain in procedural terms but did not seem to consider the rationale, and a third group seemed 
to rationalize their instructional approaches on the spot. If teachers have difficulty articulating 
their PCK, and if, as we believe, they are sometimes forming it in the moment (e.g., during an 
interview), we wonder about the implications for the validity of their knowledge. To the extent 
that their accounts are consistent with other sources of PCK, validity is not such a concern. 
However, when teachers are the only source for a particular instance of PCK, we think that 
validity is contingent upon the knowledge being revealed by several teachers. Otherwise, C- 
PCK could be based on an N of 1, and that seems contradictory to the notion of the construct. 
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Taking into account sample size and validity, we incorporated a survey approach in our method 
because of the potential for efficiency, but, as noted above, found that it was not particularly 
effective. We thought the number of individuals we collected survey information from might 
compensate for any individual’s inability to express their PCK. And although their contributions 
were somewhat complementary, we found interviews to be more productive. The researchers 
who developed the CoRe interview approach did so in large part because of their lack of success 
eliciting PCK through a questionnaire, so although we were disappointed by the ineffectiveness 
of surveys, we were not particularly surprised. The implication is that collecting PCK from 
practitioners is more labor intensive (and therefore more expensive) than we anticipated. 


Another dimension of our PCK collection efforts consists of interviews with instructional 
materials developers. Our experience with these individuals contrasts sharply with our efforts to 
gather PCK from teachers. The materials developers we interviewed were able to articulate both 
procedure and rationale, but this finding is based on the small number we were able to identify 
for SPM. We interviewed individuals we knew and asked them to recommend others, but there 
is an implicit bias in that method. For example, all of them developed what might be called non- 
traditional materials as opposed to the hardbound textbooks present in many elementary 
classrooms. 


Based on what we have learned, we find it difficult to determine whether findings are 
representative of the PCK-collection process as a whole or are dependent upon the topic. SPM 
and its newness at the elementary level have presented challenges that may be unique to such 
unfamiliar content. Consequently, we are considering applying the methodology to a third topic, 
one that is more like Interdependence in terms of its traditional place in elementary grades 
instruction. 
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PCK Codebook for Empirical Literature 


Code Description Extraction Rules Confidence Rating 
Misconceptions are student ideas that are in conflict | Extract all The confidence rating is about how confident 
with accepted scientific ideas. Misconceptions misconceptions from we are that this misconception is widespread 
typically arise from students’ interaction with the an article, even if among 5" grade students based on the study 
physical world around them. A common identifying in the article. If the point of a study was to 
misconception is that air does not have mass because | misconceptions was identify student misconceptions and the article 
they can’t “feel” it. Misconceptions are neither good | not the intent of the fares well in the rapid SoE review, the 
nor bad, but they do tend to be deeply ingrained in study. Can modify or | misconception gets a high confidence rating. 
students’ thinking. Some are part of a learning paraphrase article text | All other misconceptions get a low confidence 
progression for a topic, suggesting that many for clarity, brevity. rating. NOTE: when we synthesize across 

: : students will have the misconception at some point Include missing studies, a misconception that shows up several 
Misconception 


as they develop full understanding. 


conceptions with 
misconceptions. 
Capture related 
misconceptions 
separately when 
possible. When 
present, capture the 
cognitive source along 
with the 
misconception. 


times with a low rating may receive a high 
rating based on the accumulation of evidence. 
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Code 


Description 


Extraction Rules 


Confidence Rating 


In contrast to misconceptions, misinformation is an 
incorrect fact not derived from every day experience 
with the physical world. For example, students 
might think that water freezes at 32 degrees Celsius. 
Students’ misinformation is probably not as deeply 
ingrained in their thinking as misconceptions are. 


Extract all 
misinformation from 
an article, even if 
identifying 
misinformation was 
not the intent of the 


The confidence rating is about how confident 
we are that this misinformation is widespread 
among 5" grade students based on the study 
in the article. If the point of a study was to 
identify student misinformation and the article 
fares well in the rapid SoE review, the 


Misinformation study. misinformation gets a high confidence rating. 
All other misinformation gets a low 
confidence rating. NOTE: when we 
synthesize across studies, misinformation that 
shows up several times with a low rating may 
receive a high rating based on the 
accumulation of evidence. 

Teaching tips are pieces of advice for teachers and Extract all teaching If the point of a study was to identify teaching 
are bigger than an individual activity, things that can | tips from an article, tips and the article fares well in the rapid SoE 
be useful for teachers to know when teaching the even if identifying tips | review, the tip gets a high confidence rating. 
topic. For example, “Investigating the expansion and | was not the intent of All other tips get a low confidence rating. 

Idea-level compression of air is important for students’ the study. NOTE: when we synthesize across studies, a 

Consideration understanding of the concept of empty space tip that shows up several times with a low 

between particles.” rating may receive a high rating based on the 

NOTE: If a tip can be associated with all big ideas in accumulation of evidence. 

the topic, it should be coded as a unit-level 

consideration instead (see below). 

Unit-level considerations (ULCs) are broader than Extract all ULCs from | If the point of a study was to identify ULCs 

teaching tips and apply to the entire unit, but they an article, even if and the article fares well in the rapid SoE 

should not be broader than the unit (the latter might identifying ULCs was review, the ULC gets a high confidence 
eievel actually be pedagogical knowledge instead of PCK). | not the intent of the rating. All other ULCs get a low confidence 

Consideration For example, “Having students interact with study. rating. NOTE: when we synthesize across 


computer simulations that depict particle-level 
representations of matter can help students 
understand the particle model of matter.” Code a 
ULC to all big ideas in the topic. 


studies, a ULC that shows up several times 
with a low rating may receive a high rating 
based on the accumulation of evidence. 
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Code 


Description 


Extraction Rules 


Confidence Rating 


Prompts are questions or tasks that teachers would 
pose to their students in order to elicit their thinking 
in writing or orally to use for formative purposes. 


(Some prompts that are appropriate for research 


Extract a prompt if the 
reviewer can envision a 
teacher using it with 
students as is. Bar for 


The confidence in a prompt is based entirely 
on the content of the prompt (e.g., how well 
aligned it is with the idea, how “usable” it is 
by a teacher), as judged by the reviewer. For 


wee ; modifying prompt example, a prompt that contains wording that 
purposes (used in interviews, etc.) may be } ; : ; 
: : from article text is may be inaccessible for students would 
inappropriate for classroom use.) ; ; : : 
higher than that for receive a low confidence rating. 
Prompt ; : 
misconceptions. If 
prompts that 
accompany activities 
or activity seeds can 
stand independently, 
capture as prompts. If 
not, don’t. 
Instructional activities are stand-alone, ready-to-use | Extract an instructional | If the point of the article was to investigate the 
activities that teachers can use in their instruction “as | activity if a teacher can | impact of an instructional activity, the 
is” with no additional written materials or training. use it as is—i.e., ithas | confidence rating will be based on the 
Their purpose is to develop understanding of a big sufficient context and findings of the article and a rapid SoE. If the 
: idea. That is, the article should provide enough instructions. instructional activity was incidental, the 
Instructional information so that teachers are able to implement confidence rating will be low. If the article 
Activity the activity in their classrooms. The learning goal explicitly investigates the efficacy of an entire 


should be explicit or easily inferred. 


NOTE: Eventually, we will categorize the activities 
into more general instructional strategies, such as lab 
experiments, simulations, readings. 


unit and uses an individual activity to 
illustrate the material, the activity would 
receive a low rating. 
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Code 


Description 


Extraction Rules 


Confidence Rating 


Activity Seed 


Not a ready-to-use instructional activity, but a 
fleshed out idea for an activity. The seed must have 
enough description to determine that it fits some big 
idea(s) and to give a reasonable expectation that 
teachers could develop it into an activity. 


Do not capture if seed 
is unsuccessful in 
implementation or in 
need of substantial 
modifications in order 
to be helpful 


In order to have a high confidence rating, an 
activity seed must meet all three of the 
following criteria: 


¢ Is it explained in a way that is clear and 
accessible to teachers? 


e Are students likely to learn targeted 
content from it? 


¢ Is it feasible? (time required, materials 
required) 


Summative 
Assessment 
Activity 


Summative assessment activities are stand-alone, 
ready-to-use activities that teachers can use in their 
instruction to evaluate students. 


Extract an assessment 
if a teacher can use it 
as iS--1.e., it has 
sufficient context and 
instructions. Briefly 
summarize the form 
and substance of the 
assessment, and if 
there's a rubric, 
describe how it's 
structured, what kinds 
of factors it takes into 
account. 


If the assessment does not have reliability 
and validity info, it should receive a LOW 
rating. (NOTE: if the reliability and 
validity info are in another article, the 
assessment should be extracted from that 
article.) 


If the assessment does have reliability and 
validity info, the rating should be based 
on that information. Reliability should be 
above 0.7, and there should be at least one 
form of validity evidence. 
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Code 


Description 


Extraction Rules 


Confidence Rating 


Common student experiences are things that a 
teacher can capitalize on in instruction, knowing that 
there is a good chance that most students have 
similar experiences. For example, most 5" grade 
students will have firsthand experience with an 
inflated balloon expanding or contracting based on 


Extract a common 
student experience if 
there is evidence in the 
article that most 
students come to 
instruction with the 


The confidence rating is based on a rapid SoE 


review. 


Common : ; 
Student temperature. Most have also observed a puddle experience. An article 
; disappear over time. Common student experiences that describes what just 
Experiences ms 
may be keyed to one big idea or more than one. one student has 
experienced is not 
sufficient. Do not 
include previous 
instruction 
experiences. 
Developmental challenges are things that students Extract a The confidence rating is based on a rapid SoE 
struggle with that are broader than misconceptions. developmental review. 
For example, 5" grade students and younger may challenge if there is 
struggle to accept the existence of matter that is too evidence in the article 
small to see. Developmental challenges may be that most students 
keyed to one big idea or more than one. come to instruction 
Some developmental challenges may have associated waltie challense, 
ULCs. For example, we think that kids do not apply 
Developmental : 
explanatory frameworks consistently, but rather that 
Challenge 


it is context specific (e.g., students may understand 
the particle model in the context of boiling water, but 
will not apply it to condensation on a cold drink can). 
The unit-level consideration is that teachers can't 
assume that just because kids use the particle model 
appropriately in one context, they will use it 
appropriately in another. 
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Code 


Description 


Extraction Rules 


Confidence Rating 


Learning 
Progression 


A learning progression will probably be identified 
explicitly in an article. A learning progression is at 
the topic level, so we do not need to code to 
individual big ideas. All misconceptions in a 
learning progression can be coded with the 
progression. 


Extract a learning 
progression if the 
article describes a 
sequence of 
increasingly 
sophisticated and 
scientifically accurate 
understandings and 
skills within a domain 
that learners develop 
over several years. 


The confidence rating is based on a rapid SoE 


review. 
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